Challenges and pitfalls in malware research

As the malware research field became more established over last two decades, new questions arose, such as how to make reproducible, bring scientific rigor attack papers, or what is an appropriate dataset for relevant experimental results. The challenges these pose also brings pitfalls that affect multiple stakeholders. To help answering those and highlight potential be avoided, in this paper, we present a systematic literature review of 491 papers on published major security conferences between 2000 2018. We identified most common past propose method assessing current (and future) research. Our goal towards integrating science engineering best practices develop further, improved by learning from issues body work. far know, largest its kind first summarize methodology avoids them. In total, discovered 20 limit impact reproducibility. range (i) lack proper threat model, complicates paper’s evaluation, (ii) use closed-source solutions private datasets, report yet-to-be-overcome are inherent nature, non-deterministic analysis Based our findings, set actions taken development community future work: Consolidation constituted diverse approaches (e.g., solutions, offensive research, landscapes/observational studies, network traffic/system traces analysis); design with clearer, direct assumptions positioning proofs-of-concept vs. deployable); (iii) experiments reflects emphasizes) target scenario proposed solution corporation, user, country-wide); (iv) clearer exposition discussion limitations used technologies exercised norms/standards antiviruses ground-truth).